Cognitively Motivated Novelty Detection in Video Data Streams

Automatically detecting novel events in video data streams is an extremely challenging task. In recent years, machine-based parametric learning systems have been quite successful in exhaustively capturing novelty in video if the novelty filters are well-defined in constrained environments. Some important questions however remain: How close are such systems to human perception? Can results derived from comparing human perception with machine novelty help tasks such as storing (indexing) and retrieval of novel events in large video repositories? In this chapter a quantitative experimental evaluation of human-based vs. machine-based novelty systems is canvassed. A machine-based system for detecting novel events in video data streams is first described. The issues of designing an indexing-strategy or “Manga” (comic-book representation is termed as “manga” in Japanese) to effectively determine the “most-representative” novel frames for a video sequence are then discussed. The evaluation of human-based vs. machine-based novelty is quantified by metrics based on location of novel events, number of novel events, etc. Low-level image features were used for machine-based novelty detection and do not include any semantic processing such as object detection to keep the computational load to a minimum.